Systems and methods for segmentation and measurement of a skin abnormality

ABSTRACT

A method is proposed for identifying (“segmenting”) at least one portion of the skin of an animal which is a region of interest (e.g. a portion which is subject to an abnormality such as a tumor). The method uses at least a temperature dataset obtained by measuring the temperature of each of a plurality of points of a region of the skin. An initial segmentation may be performed using the temperature data based on a statistical model, in which each point is segmented based on its temperature and optionally that of its neighbors. The initial segmentation based on the temperature data may be improved using a three-dimensional model of the profile of the skin, and the enhanced segmentation may be used to improve the three-dimensional model.

FIELD OF THE INVENTION

The present invention relates to an imaging method and an imaging systemfor generating three-dimensional (3D) images of a region of interest onskin, e.g. a region of interest associated with a three-dimensionalabnormality such as a tumor or wound on the body of an animal,especially a mammal, such as a rat or other rodent. It further relatesto a method and system for extracting numerical data characterizing theabnormality.

BACKGROUND OF THE INVENTION

Much laboratory research involves studying growths and/or wounds on theskin of a laboratory animal such as a rat or other mammal. Inparticular, subcutaneous growths such as tumors are often studied. Forexample, in the case of a laboratory animal which is subject to atreatment regime, measurements of the extent and/or the growth speed oftumors give useful information about the treatment regime. The tumorsmay be measured laterally (that is, their extent parallel to the skinsurface) or by their protrusion (that is, their extent perpendicular tothe skin surface). Other research involves measurement at intervals ofwounds on the skin of a laboratory animal, i.e. cavities in the skin,e.g. to measure how quickly wounds heal (or expand).

Conventionally, measurements of growths/cavities are obtained manuallyusing calipers, often after the animal has been shaved. This has severaldisadvantages: it is subject to human error; and it is somewhatsubjective since different laboratory workers may measure tumors inslightly different ways (e.g. measuring different positions on thetumor), and may apply different levels of compression to the tumor usingthe calipers. The measurement process may be time-consuming and haveinsufficient repeatability.

Recently methods have been proposed for automatically obtaining athree-dimensional model of the animal's skin. The profile of the skin(that is, the three-dimensional shape of the surface of the skin) isobtained by methods such as laser scanning, photometry or stereoscopy.However, it is technically challenging to process the resulting dataset,because it is hard to determine how much of the shape of the model isdue to the abnormality and how much is due to the natural curvature ofthe skin of the animal.

SUMMARY OF THE INVENTION

In general terms, the present invention proposes identifying at leastone portion of the skin of an animal which is a region of interest (e.g.an abnormality) using at least a temperature dataset obtained bymeasuring the temperature of each of a plurality of points of a regionof the skin. The identification process may be referred to as“segmentation”, i.e. segmenting (classifying) the skin into the at leastone portion of the skin which is part of a region of interest, and otherportion(s) of the skin which are not part of the region of interest.This concept is based on the observation that certain skin abnormalitieschange the distribution of temperatures on the skin. For example, formany tumors the temperature of the skin above the tumor is lower thanthat of surrounding skin.

The region of interest may be any portion of the skin which is expectedto be at a different temperature from a neighboring portion of the skin.For example, it may be a portion having a high or low proportion of haircompared to the neighboring portion of the skin. However, moretypically, it is a portion which is subject to an abnormality whichchanges the temperature of the skin, e.g. a tumor, or possibly a wound.

The temperature dataset is preferably used in combination with a modelof the three-dimensional profile of the skin. In this respect thetemperature dataset is “fused” with the three-dimensional model of theprofile of the skin, e.g. to obtain an enhanced segmentation of the skinand/or an improved three-dimensional model. The model of the skin may,for example, be obtained by combining images of the region of the skincaptured using a plurality of cameras. For example, the images may becombined using stereoscopy to create the model.

In one example, an initial segmentation may be obtained based on thetemperature dataset (for example as explained below), and may beenhanced using the three dimensional model of the profile of the skin.For example, if the initial segmentation identifies a certain portion ofthe skin as being part of the region of interest, but a correspondingportion of the model of the three-dimensional profile of the skin doesnot meet a criterion indicative of the region of interest (for example,if the portion of the model obeys a continuity criterion with respect toanother neighboring or surrounding portion of the skin), the portion ofthe skin may be re-classified as not being part of the region ofinterest.

In another example, which can be combined with the first, a segmentationobtained using the temperature data, is used to improve at least a partof the three-dimensional model of the profile of the skin. For example,the improvement may improve a defective portion of the three-dimensionalmodel, such as a portion which is missing, or which is determined tomeet a criterion indicative of being low accuracy. Specifically, theimprovement may add an interpolation surface to at least part of thedefective portion of the three-dimensional model of the profile of theskin. The interpolation surface may have a first edge which, accordingto the segmentation based on the temperature data (e.g. the enhancedsegmentation), is at edge of the portion of the skin which is part ofthe region of interest. At a second edge of the interpolation surface,which according to the segmentation is in a portion of the skin which ispart of the region of interest, the interpolation surface may becontinuous with, and optionally may have a gradient equal to that of,the three-dimensional model at the other side of the second edge.

One or more numerical parameters characterizing the region of interestmay then be derived (e.g. automatically) from the modifiedthree-dimensional model of the profile of the skin, e.g. a valueindicative of the volume of an abnormality associated with the region ofinterest. For example, the numerical parameter(s) may comprise a volumebetween the modified three-dimensional model and a baseline surface,which is an estimate of what the surface of the skin would have been inthe absence of the abnormality.

As mentioned above, an initial segmentation of the portion of the skinwhich is part of the region of interest may be formed using thetemperature data. Specifically, the initial segmentation of the skinbased on the temperature dataset may be based on whether each point ofthe region of the skin has a temperature, according to the temperaturedataset which is above or below a cut-off temperature. Note that asexplained below, the cut-off temperature may be the same for all skinpoints, or may be different for different respective points of the skin.The cut-off temperature(s) are derived from the temperature dataset.

For example, the cut-off temperature(s) may be derived from astatistical model of statistical variation of temperature within aregion of the skin containing both skin which is part of the region ofinterest and skin which is not part of the region of interest. Thecut-off temperature for a given point of the skin may be a temperatureat which the point of the skin is equally likely according to thestatistical model to be part or not part of the region of interest.

The statistical model may be characterized by a first temperature valueindicative of an average (e.g. mean) temperature of skin points whichare part of the region of interest, and a second temperature valueindicative of an average (e.g. mean) temperature of skin points whichare not part of the region of interest. The statistical model may befurther characterized by a first variance value indicative of atemperature variability of skin points which are part of the region ofinterest, and a second variance value indicative of a temperaturevariability of skin points which are not part of the region of interest.In other words, according to the statistical model, the likelihood thatany given point of the skin is part of the region of interest is also afunction of the first and second temperature values, and optionally alsothe first and second variance values.

Furthermore, according to the statistical model, the likelihood that anygiven point of the skin is part of the region of interest may also be afunction of the temperatures, according to the temperature data, of oneor more other points on the skin which each meet a proximity criterionwith respect to the given point. For example, the proximity criterionmay be that the other point on the skin is within a certain distance ofthe given point. Thus, for any given skin point, the proximity criteriondefines a neighborhood consisting of other skin points, and the cut-offtemperature for the given skin point depends upon the temperature of theother skin points in the neighborhood.

For example, if the first temperature value is higher (or alternativelylower) than the second temperature value, according to the statisticalmodel, the likelihood that a given skin point is part of the region ofinterest is an increasing (decreasing) function of the respectivetemperatures of the skin points in the corresponding neighborhood. Inother words, according to the statistical model, the given skin point ismore likely to be part of the region of interest if its neighboringpixels are warmer (colder).

The result of defining the statistical model in this way is that theeffect of noise in the temperature dataset is reduced. This is becausethe temperature of a given point has to differ from the temperature ofits neighboring points by a higher amount in order for the given pixelto be classified differently from its neighbors.

Optionally, an iterative procedure may be carried out in which, in eachof a plurality of steps, an current estimate of one of more numericalparameters of the statistical model (e.g. the first and secondtemperature values, and/or the first and second variances) is used toperform a segmentation of the region of the skin, and the segmentationis used to produce an improved estimate of the numerical parameter(s).

The invention may be expressed in terms of a method or system forprocessing captured data relating to the region of the skin, for exampleto perform the segmentation, and/or to derive the numerical parameter(s)characterizing the abnormality. Alternatively, the invention may beexpressed as a computer program product (e.g. stored in non-transitoryform on a tangible recording medium) comprising program instructionsoperative when implemented by a processor, to perform the method.Alternatively, the invention may be expressed as an imaging method orsystem which captures the data relating to the region of the skin, andthen processes it by the method.

BRIEF DESCRIPTION OF THE DRAWINGS

An embodiment of the invention will now be described for the sake ofexample only with reference to the following figures in which:

FIG. 1 is a schematic view of an imaging system which is an embodimentof the present invention;

FIG. 2 is a flow diagram of a method which is an embodiment of theinvention;

FIG. 3 illustrates a first possible statistical model used in the methodof FIG. 2;

FIG. 4(a) shows a preliminary segmentation, produced in the method ofFIG. 2, of a region of skin, to estimate a portion of the skin which issubject to an abnormality, and FIG. 4(b) shows an improved segmentationwhich may be produced from the preliminary segmentation;

FIG. 5 illustrates a second possible statistical model used in themethod of FIG. 2;

FIG. 6 is composed of FIG. 6(a) which is a three-dimensional model ofthe profile of the skin, and FIG. 6(b) which shows an enhancedsegmentation produced by fusing an initial segmentation produced fromtemperature data, such as the segmentation of FIG. 4(b), with thethree-dimensional model;

FIG. 7 illustrates an interpolation of the three-dimensional model ofthe skin profile performed in the method of FIG. 2; and

FIG. 8 illustrates the structure of a data processing system of theimaging system of FIG. 1.

DETAILED DESCRIPTION OF THE EMBODIMENTS

FIG. 1 shows schematically an imaging system which is an embodiment ofthe invention. The imaging system includes an imaging region where atleast part of an animal 1 may be placed. The animal 1 may be hand-heldor retained by a retention mechanism. The imaging region is in the fieldof view of one or more cameras 2 a, 2 b. The camera(s) producerespective two-dimensional images of the field of view, from differentrespective imaging positions and optionally in different respectiveviewing directions. When an animal 1 is located in the field of view,each of the images captured by the cameras 2 a, 2 b comprises an imageof a region of the skin of the animal 1. The animal 4 is typicallypositioned such that the region of skin contains the whole of a skinabnormality, such as a sub-cutaneous tumor or a wound.

Although the number of cameras is illustrated in FIG. 1 as two, theremay be more or fewer cameras. Furthermore the cameras 2 a, 2 b may beoperative to capture still images and/or video images, with both sortsof images being referred to as “images” in this document. Theelectromagnetic radiation captured by the camera(s) 2 a, 2 b may be inany frequency range, not necessarily the visible frequency range.

The outputs of the cameras 2 a, 2 b are transmitted to a data processingsystem 3. The construction of the data processing system 3 is explainedbelow in more detail with reference to FIG. 8. The data processingsystem 3 is operative, using the images captured by the cameras 2 a, 2b, to construct a three-dimensional numerical model of the region of theskin of the animal 1. In one possibility, the three-dimensional modelmay be constructed by stereoscopy. In a further possibility, thethree-dimensional image may be constructed using photometry, for exampleusing the photometric technique employed in WO 2009/122200, thedisclosure of which is incorporated herein by reference. In a furtherpossibility, photometry and stereoscopy may be combined. Furthermore,the cameras 2 a, 2 b may be replaced (or supplemented) with e.g. a laserimaging system, or ultrasound imaging system, which measures reflectionsof a beam moved (e.g. in a raster) to successive points of the skinregion, and forms a depth image of the region of the skin from themeasurement results.

The imaging system further comprises a thermal imaging system 4(typically an infra-red camera) which is operative to capture a thermalimage (temperature dataset) indicating the respective temperatures of aplurality of points of the region of the animal's skin. Thus, thetemperature dataset comprises a two-dimensional thermal image of theregion of the skin. The temperature dataset is transmitted to the dataprocessing system 3.

Optionally, e.g. to reduce noise, any of the thermal imaging system 4and/or the cameras 2 a, 2 b may capture multiple images at differentrespective times and combine them with each other (e.g. by averaging).Thus, for example, the temperature dataset referred to above may in factbe formed by averaging multiple thermal images captured by the thermalimaging system 4 at different respective times, optionally with aselected alignment of the images to compensate for movement of theanimal 1 in the time periods between the times that the respectivethermal images were captured.

The temperature dataset may for example be a respective temperaturevalue for each of a two-dimensional array of points (pixels) whichcorrespond under a first mapping to an array of respective pointsspanning the region of the skin of the animal. Thus, the first mapping(which depends on the position of the thermal imaging system relative tothe region of the skin) maps the curved region of the skin to a (flat)two-dimensional space in which the thermal image is defined.

Similarly, each of the images captured by the camera(s) 2 a, 2 b is aset of intensity values (optionally for each of a plurality of colors)for each of a two-dimensional array of pixels. The pixels correspond,under a respective second mapping for each camera, to points of theregion of the skin. Thus, a small sub-area of the typically curvedregion of the skin (e.g. what is referred to below as “a point of theskin”) corresponds under the first mapping to a first number of pointsin the thermal image, and under each of the second mappings to arespective second number of points in the respective images captured bythe camera(s) 2 a, 2 b. The ratios of the first number and the secondnumbers depend upon the resolutions of the cameras 2 a, 2 b and thethermal imaging system 4. In this document, the temperature of a “pointof the skin” may refer to a mean of the temperatures of the points inthe temperature dataset which correspond to the point of the skin underthe first mapping. For simplicity, the following explanation refers to“segmenting the region of skin” (i.e. assigning portions of it to acorresponding one of multiple classes), and the segmentation referred tois to be understood as being performed in any one of these correspondingdiscrete two-dimensional spaces, or in yet another discretetwo-dimensional space which corresponds by another mapping to thesurface of the skin. For example, conveniently the segmentation may beperformed in the two-dimensional space in which the temperature datasetis defined, or in a discrete two-dimensional space having a lowerresolution than the thermal image and the images captured by the cameras2 a, 2 b.

Although the thermal imaging system 4 is illustrated as being separatefrom the camera(s) 2 a, 2 b which capture the images used to constructthe three-dimensional model of the profile of the region of the skin, inother embodiments the thermal image itself might be used to generate thethree-dimensional model. For example, the imaging system might comprisea plurality of thermal imaging systems 4 producing respective thermalimages from different respective imaging positions, and stereoscopymight be performed on the plurality of thermal images to produce thethree-dimensional model of the profile of the skin.

Turning to FIG. 2, a method 100 is illustrated which is performed by theimaging system of FIG. 1, and which is an embodiment of the method.

In step 101, the thermal imaging system 4 is used to capture thetemperature dataset (a temperature map), and the camera(s) 2 a, 2 b areused to capture other images of the region of the skin of the animal.Thus, step 101 is carried out by the thermal imaging system 4 and thecameras 2 a, 2 b under the control of the data processing system 3.

The remaining steps of method 100 are performed by the data processingsystem 3 alone. In step 102 numerical parameter(s) of the temperaturedistribution are obtained from the temperature dataset. These mayinclude a cut-off temperature used in step 103 to perform segmentation.

For example, in step 102, a histogram may be plotted of the number ofthe pixels of the temperature data against temperature (that is, thehistogram indicates, for each of a sequence of non-overlappingtemperature ranges, the respective number of points (pixels) of the skinregion having a temperature in that range). The result would ideally beas shown in FIG. 3. That is, there are may be two peaks, with a dipbetween them. This is because it is expected that points which aresubject to the abnormality (e.g. subject to a tumor, or any otherabnormality which changes the temperature of the skin) would have atemperature which is distributed as a Gaussian distribution with a firstmean temperature value (denoted μ₁) and a first variance value (denotedσ₁), and that points which are not subject to the abnormality would havea temperature which is distributed as a Gaussian distribution with asecond mean temperature value (denoted μ₂) and a first variance value(denoted σ₂). The values of the first and second temperature values μ₁and μ₂, and the first and second variance values σ₁ and σ₂ may not beknown in advance (and indeed it may not be pre-known which of the firstand second temperature values is higher, and/or which of the first andsecond variance values is higher). The histogram in FIG. 3 is thuslikely to be approximately a sum of two Gaussians curves, each having anamplitude which depends of the proportion of the region of the skinwhich is subject to the abnormality. Note that due to noise in thetemperature data the true first and second temperature values μ₁ and μ₂will not correspond exactly to the maxima in the histogram of FIG. 3.Nevertheless, the value of μ₁ and μ₂ may be approximated as the twotemperatures corresponding to the peaks of the histogram of FIG. 3.Similarly, the variance values variance values σ₁ and σ₂ may beapproximated from the respective widths of the peaks.

In an ideal case, the cut-off temperature could be chosen based on aminimum point of the distribution, as shown in FIG. 3, between thepeaks. However, in reality, the histogram is likely to contain noise, sothat the distribution has multiple local minima between the peaks. Onesuch minimum may be selected at random as the cut-off temperature, orthe cut-off temperature may be set to be an average of the temperaturescorresponding to the two peaks of the distribution (which may or may notthemselves be well defined, depending upon the noise).

In step 103, using the temperature data and the numerical parameters(e.g. the cut-off value), the region of the skin is segmented. This maybe done by determining points on the skin for which the temperatureaccording to the temperature dataset is above or below the cut-offtemperature.

Optionally, the cut-off temperature may be selected based on astatistical model of skin temperatures similar to that shown in FIG. 3,and based on numerical values (such as σ₁, σ₂, μ₁ and μ₂) which areestimated from the histogram of FIG. 3. For example, the cut-offtemperature may be the temperature such that, according to thestatistical model, the probability is 50% that a portion of the skin atthat temperature is subject to the abnormality. Since the cut-offtemperature depends upon the first and second temperature values μ₁ andμ₂, and the variance values σ₁ and σ₂, it can be said that each pixel ofthe skin region is segmented based on a statistical model which is afunction of the temperature of the pixel according to the temperaturedataset, and also the first and second temperature values μ₁ and μ₂, andthe variance values σ₁ and σ₂. This can be referred to as an“expectation maximization” algorithm.

The result of applying this cut-off temperature for the segmentation maybe as shown in FIG. 4(a). Here the outer border of the region of theskin under consideration is indicated as 20. As shown in FIG. 4(a) anumber of areas of the skin 21, 22, 23 a, 23 b have a temperaturethroughout which is above (or below) the cut-off temperature, while area25, and also the area 24 which extends to the outer border 20 of theskin region 20, have a temperature which is below (or above) the cut-offtemperature. The areas 23 a, 25 are clusters of pixels, while the areas23 b are individual pixels. Depending upon whether the temperature ofthe region 24 is above or below the cut-off temperature, it can bedetermined which of the temperature values μ₁ and μ₂ is higher, if thisis not pre-known. Note that, for a tumor, the first temperature value μ₁of the portion of the region of the skin which is subject to the tumor(i.e. overlies the tumor) may optionally be assumed to be lower than thesecond temperature value μ₂ of the portion of the region of the skinwhich is not subject to the tumor. Note that in FIG. 4, the small areas23 a, 23 b are likely to be the result of noise in the temperaturedataset, e.g. due to blood vessels or hair on the surface of the animal,while one of the larger areas 21, 22 is likely to represent the tumoritself, at least approximately. The area 25 has a temperature on theother side of the cut-off temperature from the rest of the area 21,which surrounds it. Therefore, if the area 21 approximately representsthe abnormality, the area 25 may well be a noise artifact (e.g. due tohair or a blood vessel on the tumor).

Optionally, in step 103 the segmentation may be performed using a moresophisticated statistical model, referred to here as a “modifiedexpectation maximization” (MEM) model. MEMs were proposed in “Anadaptive segmentation and 3-D visualization of the lungs” by J.Dehmeshki in Pattern Recognition Letters 20 (1999) 919-926, thedisclosure of which is incorporated by reference, which dealt with theunrelated technical field of delineating lungs within computerizedtomography images. According to this more sophisticated statisticalmodel, the likelihood that any given point of the skin is subject to theabnormality may additionally be a function of the temperatures,according to the temperature dataset, of one or more other points on theskin which each meet a proximity criterion with respect to the givenpoint. For example, the proximity criterion may be that the other pointon the skin is within a certain distance of the given point. Thus, forany given skin point, the proximity criterion defines a neighborhoodconsisting of other skin points. The measure of distance may forexample, be Euclidean distance in a two-dimensional space correspondingto the skin surface, or some other measure of distance, such as theManhattan distance in the two-dimensional space.

To put this another way, each pixel of the skin region is segmentedbased on a statistical model in which the probability that it is (or isnot) overlying the tumor is a function of (i) its own temperature, (ii)the first and second temperature values μ₁ and μ₂, and the first andsecond variance values σ₁ and σ₂, and (iii) the measured temperature ofthe neighboring pixels. The more sophisticated statistical modelincorporates prior knowledge that points of the skin which are subject(or not subject) to the tumor have a high probability of containingother such points within their neighborhood. Thus, even if a given pointhas a temperature which is not associated with the abnormality, thepoint still has a high chance of being subject to the abnormality if itis neighbored by (e.g. is surrounded by) other points subject to theabnormality. The exact form for the statistical model (as given in Eqns.(5) and (6) of the above-referenced publication by J. Dehmeshki) is an aposteriori probability of the given point being in either of the twoclasses (i.e. subject to the abnormality or not) given the values σ₁,σ₂, μ₁ and μ₂, and the temperatures of the other points in itsneighborhood. The cut-off temperature for the point on the skin is suchthat the a posteriori probability is 50% that the point is subject tothe abnormality.

For example, considering the case that μ₁ is lower than μ₂, according tothe statistical model the likelihood that a given skin point is subjectto the abnormality may be a decreasing function of the respectivetemperatures of the skin points of the corresponding neighborhood. Inother words, according to the statistical model, the given skin point ismore likely to be subject to the abnormality if its neighboring pixelsare colder.

The size of the neighborhood (i.e. the proximity criterion) may bechosen with prior knowledge of the abnormality. For example, if it isbelieved that the abnormality will be at least 10 pixels wide, theneighborhood may be chosen to have approximately this diameter. Toexpress this more generally, the neighborhood is characterized by anextent (a distance parameter) which is based on prior knowledge of theassociated normality.

The result of defining the statistical model in this more sophisticatedway is that noise in the thermal model is reduced. This is illustratedin FIG. 4(b) which shows schematically the result of using the MEMtechnique on the skin region 20. The effect of noise is reduced (i.e.areas 23 a, 23 b and 25 are eliminated) because the temperature of agiven point in the temperature map (i.e. an individual pixel of thetemperature map, or small cluster of pixels) has to differ from thetemperature of its neighboring points by a higher amount in order forthe given point to be classified differently from its neighbors.

Note that an alternative to performing the MEM using posteriorprobabilities is to downsample the temperature map (i.e. reduce itspixel resolution such that a single pixel of the downsampled map has atemperature which is an average of a respective neighborhood (aplurality of pixels) in the original temperature map), and comparingeach pixel in the downsampled temperature map to the cut-offtemperature. That is, the original temperature map may be downsampled toproduce a downsampled map (e.g. using a multi-scale approach any numberof times), and then each pixel of the downsampled temperature map iscompared to the cut-off temperature. The segmentation is done based onthe result of the comparison. Thus, the proximity criterion in this caseis whether, following to the downsampling, two points of the originaltemperature map are mapped to the same pixel of the downsampled map bythe downsampling operation. For example, if the original temperature maphas a pixel resolution of 1024×1024, the depth map may be downsampled to512×512, 256×256, 128×128, or 64×64. Compared to using theprobabilisitic approach to MEM, the downsampling approach requires lesscomputational effort. The prior knowledge of the abnormality can be usedto select the amount of downsampling applied to the original temperaturemap.

Once this improved segmentation has been performed, the respectivetemperatures of the pixels corresponding to skin points which, accordingto the segmentation, are subject to the abnormality, may be used toproduce improved values for the first temperature value μ₁ and the firstvariance value σ₁. Similarly, the respective temperatures of the pixelscorresponding to skin points which, according to the segmentation, arenot subject to the abnormality, may be used to produce improved valuesfor the second temperature value μ₂ and the first variance value σ₂. Thesegmentation process using the more sophisticated statistical model(i.e. using the temperatures of neighboring pixels) can then be repeatedusing the improved values of σ₁, σ₂, μ₁ and μ₂.

Optionally, step 103 may include an iterative procedure in which, ineach of a plurality of steps, (i) an current estimate of one of morenumerical parameters of the statistical model (e.g. σ₁, σ₂, μ₁ and μ₂)is used to perform a segmentation of the region of the skin based on themore sophisticated statistical model employing respective neighborhoodsfor each pixel, and (ii) the segmentation is used to produce an improvedestimate of the numerical parameter(s).

The computational burden of performing this process may be high.Optionally, it can be reduced by defining the statistical model on theassumption that, instead of the sum of two Gaussian distributions shownin FIG. 3, the distribution of the temperatures of the pixels is asshown in FIG. 5. Here the distribution contains a first triangularportion defined based on σ₁ and μ₁, and a second triangular portionbased on σ₂, and μ₂. The two triangular portions may meet at a pointwhich is used as the cut-off temperature.

In step 104 of method 100, the image data captured by the cameras 2 a, 2b is used to produce a three-dimensional model of the region of theskin, e.g. by one of the conventional methods described above, such asstereoscopy. In step 105 any portions of the model which are defective,e.g. missing or not reliable (i.e. which meet a criterion indicative ofnot being reliable), may be identified. A portion of the model may bedefective for one of several reasons. One, explained below withreference to FIG. 7, is that the corresponding portion of the skin wasmissing in at least one of the images captured by the cameras 2 a, 2 bdue to occlusion. In this case, the defective portion of the model maybe a gap in the model. Alternatively, the defective portion of the modelmay be present but may not be reliable, because it constructed based oninadequate data for it to be reliable. For example, each camera 2 a, 2 bmay not be able to take reliable images of any portions of the skinrange 20 which are farther from the camera 2 a, 2 b than a certaindistance.

For example, the three dimensional model of the profile of the skin inregion 20 (i.e. the skin for which the temperature dataset is available)is illustrated in FIG. 6(a) as a contour map. The three-dimensionalmodel comprises a portion 30 which is not subject to the abnormality.This portion 30 may be identified from the three-dimensional modelaccording to a continuity criterion. For example, the portion 30 may besubstantially planar. In another example, the portion 30 may be (gently)curved but with a curvature which is both below a certain threshold andspatially uniform (to within a certain tolerance) throughout the portion30. A baseline surface can be formed extending throughout the region 20.The baseline surface includes the portion 30 and additionally one ormore portions which are formed by interpolation using the portion 30(i.e. smoothly filling any gaps in the portion 30). The baseline surfaceis an estimate of how the skin profile would have been if theabnormality had not existed. The three-dimensional model furthercomprises a portion 31 which does not meet the continuity criterion, andwhich is thus likely to correspond approximately to the abnormality(tumor). In FIG. 6(a), the portion 31 is drawn including contoursindicating lines on the three-dimensional model which differ from thebaseline surface by equal amounts. A portion 32 of the three-dimensionalmodel is identified as being defective (missing or unreliable). Thedefective portion 32 has a first edge 34 which meets the portion 30, anda second edge 33 which meets the portion 31.

Steps 104 and 105 may optionally be performed together as a single step.

In step 106, the three-dimensional model is used to improve asegmentation obtained in step 103. Thus, the temperature dataset is“fused” with the model of the three-dimensional profile of the skin, toobtain an enhanced segmentation of the skin. For example, comparingFIGS. 4(b) and 6(a) which represent the same skin region, the region 22of FIG. 4(d) corresponds to a part of FIG. 6(a) which is within the area30. Thus, the region 22 can be reclassified as not being subject to theabnormality. This gives a final segmentation as shown in FIG. 6(b).

In step 107, the three-dimensional model of the profile of the skinregion is modified (improved) in at least part of the defective portionof the three-dimensional model.

Before explaining step 107, we explain FIG. 7, which is across-sectional view of the three-dimensional skin model in the planemarked as A in FIG. 6(a). FIG. 7 shows also the positions of the cameras2 a, 2 b. As mentioned above, the three-dimensional model was obtainedby stereoscopy using images from the cameras 2 a, 2 b. FIG. 7 shows theportion 30 of the three-dimensional model meeting the continuitycriterion. The baseline surface extends across the whole of the region20. It comprises the portion 30 of the three-dimensional model whichmeets the continuity criterion and which includes a gap, correspondingto the portions 31 and 32 of the three-dimensional model. The baselinesurface also comprises a portion 70 which is formed by smoothlyinterpolating the portion 30 across the gap. The portion 31 of thethree-dimensional image corresponds to a portion 71 of the skin whichoverlies (i.e. is subject to) a tumor, and which is well-imaged becauseit is in the field of view of both cameras 2 a, 2 b.

From FIG. 7, it can be understood why a defective portion of thethree-dimensional model exists. Although all the surface of skinprotrusion is in the field of view of camera 2 a, the defective portion32 is not in the field of view of the camera 2 b due to occlusion, sostereoscopy cannot be used to construct the portion 32 of thethree-dimensional model. Specifically, the defective portion 32 extendsbetween a line 33 where the line of sight 73 of the camera 2 bintercepts the skin surface 71 (in FIG. 7, this line 33 appears as apoint, where the line 33 intercepts the plane of the figure), and theline 34 where the line of sight 73 intercepts the baseline surface 30,70 (again, in FIG. 7, the line 34 appears as a point, where the line 34intercepts the plane of the figure). According to the segmentationmodel, there is a line in the defective portion 32 of thethree-dimensional model which is the edge of the portion of the skinsubject to the tumor. This line appears in FIG. 7 as the point 75, wherethe line intercepts the plane of the figure.

In step 107, the three-dimensional model is supplemented by adding to itan interpolation surface 76 which appears in FIG. 7 as a line, where theinterpolation surface 76 intercepts the plane of the figure. Theinterpolation surface 76 has a first edge which is the edge 75 of theportion of the skin which is subject to the abnormality according to thesegmentation obtained in step 106. This first edge 75 of theinterpolation surface 76 lies on the baseline surface 30, 70. Theinterpolation surface 76 has a second edge which is the line 33. Thus,the added interpolation surface 76 is a good estimate of the surface ofthe skin within the defective portion 32 of the three-dimensional model.

The added interpolation surface 76 of the three-dimensional model iscontinuous with the surface 71 at the line 33. Furthermore, preferablythe gradient (in three-dimensions) of the added surface 76 is equal tothat of the surface 71 at the line 33. In other words, the gradient ofthe surface 71 at the line 33 (which can be obtained reliably, sincesurface 71 is reliable) is used to set the gradient of the interpolationsurface 76 at the line 33.

In step 108, one or more numerical parameters characterizing theabnormality are derived (e.g. automatically) from the modifiedthree-dimensional model of the profile of the skin, e.g. a valueindicative of its volume. For example, the numerical parameter(s) maycomprise a calculated volume between the portion of the modifiedthree-dimensional model representing the skin over the tumor (i.e. thesurfaces 71, 76), and the baseline surface 30,70 of the skin, whichrepresents the skin as it would have been in the absence of theabnormality.

FIG. 8 is a block diagram showing a technical architecture of the dataprocessing system 3. The data processing system 3 includes a processor322 (which may be referred to as a central processor unit or CPU) thatis in communication with the image capture devices 2 a, 2 b, 4 forcontrolling when they capture images and for receiving the images.

The processor 322 is also in communication with memory devices includingsecondary storage 324 (such as disk drives or memory cards), read onlymemory (ROM) 326, and random access memory (RAM) 3210. The processor 322may be implemented as one or more CPU chips.

The system 300 includes a user interface (UI) 330 for controlling theprocessor 322. The UI 330 may comprise a touch screen, keyboard, keypador other known input device. If the UI 330 comprises a touch screen, theprocessor 322 is operative to generate an image on the touch screen.Alternatively, the system may include a separate screen 301 fordisplaying images under the control of the processor 322.

The secondary storage 324 typically comprises a memory card or otherstorage device and is used for non-volatile storage of data and as anover-flow data storage device if RAM 3210 is not large enough to holdall working data. Secondary storage 324 may be used to store programswhich are loaded into RAM 3210 when such programs are selected forexecution.

In this embodiment, the secondary storage 324 has an order generationcomponent 324 a, comprising non-transitory instructions operative by theprocessor 322 to perform various operations of the method of the presentdisclosure. The ROM 326 is used to store instructions and perhaps datawhich are read during program execution. The secondary storage 324, theRAM 3210, and/or the ROM 326 may be referred to in some contexts ascomputer readable storage media and/or non-transitory computer readablemedia.

The processor 322 executes instructions, codes, computer programs,scripts which it accesses from hard disk, floppy disk, optical disk(these various disk based systems may all be considered secondarystorage 324), flash drive, ROM 326, or RAM 3210. While only oneprocessor 322 is shown, multiple processors may be present. Thus, whileinstructions may be discussed as executed by a processor, theinstructions may be executed simultaneously, serially, or otherwiseexecuted by one or multiple processors.

Whilst the foregoing description has described exemplary embodiments, itwill be understood by those skilled in the art that many variations ofthe embodiment can be made within the scope of the attached claims. Forexample, in the explanation of the embodiment given above, the skin ofthe animal exhibits a tumor, but the embodiment is equally applicable toa case in which the skin of the animal instead exhibits a wound. Also,certain steps of the method 100 may be performed in a different orderand/or omitted. Furthermore, the method may optionally be performed by adistributed computer system including multiple processing units whichmutually communicate over a communication network. The term “based on”is used in this document such that, if a process is said to be based oncertain data, the process uses that data but may use other data also.

1. A method of identifying at least one portion of the skin of an animalwhich is a region of interest, the method comprising: receiving atemperature dataset indicating the temperature of each of a plurality ofpoints in the region of the skin of the animal; and based on thetemperature dataset, deriving an estimate of the portion of the regionof the skin which is part of the region of interest.
 2. A methodaccording to claim 1 in which the step of deriving the estimate of theportion of the skin which is part of the region of interest furtheremploys a three-dimensional model of the profile of the skin.
 3. Amethod according to claim 2 in which the temperature dataset is used toobtain an initial segmentation of the region skin which classifies theregions of the skin into at least one portion of the skin which is partof the region of interest and at least one portion which is not part ofthe region of interest, and the three dimensional model of the profileof the skin is used to enhance the initial segmentation by identifyingat least one area of the initial segmentation which incorrectlyclassifies skin as part of the region of interest, and correcting theidentified incorrect classification.
 4. A method according to claim 2further comprising: identifying at least one defective portion of thethree-dimensional model; and modifying the three-dimensional model byadding to it an interpolation surface in at least part of the defectiveportion of the three-dimensional model, the interpolation surface havinga first edge which, according to a segmentation based on the temperaturedata, is at an edge of a portion of the skin which is part of the regionof interest.
 5. A method according to claim 4 in which the interpolationsurface has a second edge in a portion of the skin which is part of theregion of interest according to the segmentation, the interpolationsurface being continuous at the second edge with the three-dimensionalmodel.
 6. A method according to claim 5 in which, at the second edge,the interpolation surface has a gradient equal to the gradient of thethree-dimensional model.
 7. A method according to claim 4 furtherincluding deriving one or more numerical parameters characterizing avolume of an abnormality associated with the region of interest from themodified three-dimensional model of the profile of the skin.
 8. A methodaccording to claim 1 in which the step of deriving an estimate of theportion of the region of the skin which is part of the region ofinterest, comprises using the temperature dataset to derive one or morecut-off temperatures, and generating a segmentation of the region of theskin, by determining whether, for each of a plurality of points in theregion of the skin, the temperature of the point according to thetemperature dataset is above or below one derived cut-off temperature.9. A method according to claim 8 in which the one or more cut-offtemperatures are derived from a statistical model of statisticalvariation of temperature within a region of the skin containing bothskin which is part of the region of interest and skin which is not partof the region of interest, the cut-off temperature for a given point onthe skin being a temperature at which the point of the skin is equallylikely according to the statistical model to be subject or not part ofthe region of interest.
 10. A method according to claim 9 in which thestatistical model is characterized by a first temperature valueindicative of an average temperature of skin points which are part ofthe region of interest, and a second temperature value indicative of anaverage temperature of skin points which are not part of the region ofinterest.
 11. A method according to claim 10 in which the statisticalmodel is further characterized by a first variance value indicative of atemperature variability of skin points which are part of the region ofinterest, and a second variance value indicative of a temperaturevariability of skin points which are not part of the region of interest.12. A method according to claim 9 in which, according to the statisticalmodel, the likelihood that any given point of the skin is part of theregion of interest is further based on the temperatures, according tothe temperature data, of one or more other points on the skin which eachmeet a proximity criterion with respect to the given point, whereby thederived cut-off temperature for the given point depends upon thetemperatures of the corresponding other points.
 13. A method accordingto claim 9, which comprises an iterative procedure in which, in each ofa plurality of steps: a current estimate of one of more numericalparameters of the statistical model is used to perform a candidatesegmentation of the region of the skin, and the candidate segmentationis used to produce an improved estimate of the one or more numericalparameters.
 14. A method according to claim 1 further comprisingcapturing the temperature data.
 15. A system comprising a processor anda data storage device storing program instructions operative, whenimplemented by the processor, to cause the processor to identify atleast one portion of the skin of an animal which is a region of interestby: receiving a temperature dataset indicating the temperature of eachof a plurality of points in the region of the skin of the animal; andbased on the temperature dataset, deriving an estimate of the portion ofthe region of the skin which is part of the region of interest.
 16. Asystem according to claim 15 further comprising a thermal imaging systemfor generating the temperature dataset.